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IN THE MICHIGAN ASSESSMENT PROJECT 
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HE present research stems directly 

from the University of Michigan 
Assessment of Clinical Psychologists 
Project which is being conducted under 
contract with the Veterans Administra- 
tion. This is a large-scale, joint research 
program designed to determine experi- 
mentally the necessary qualifications of 
a clinical psychologist. In its very design 
it offers numerous subsidiary opportuni- 
ties to test the validity of the techniques 
employed in the project. One of these 


‘THe UNiversiry OF MICHIGAN PRoj- 
ECT IN THE SELECTION OF CLINICAL 
PsYCHOLOGIsTsS! 


Ta present description will be 
limited to the Ann Arbor program 
carried out during the summer of 1947, 
since the data which serve as the basis of 
the study come from that phase of the 


‘This was a five-year study conducted under 
contract with the Veterans Administration dating 
from September 1946, under the direction of 
Prof. E, Lowell Kelly, Project Director. 


INTRODUCTION 


CHAPTER II 


PROCEDURES AND METHODS 


techniques is the Rorschach method, 
which is the one studied in this investiga- 
tion. 

The purposes of this research are: (a) 
to demonstrate that by using data ob- 
tained by means of the Rorschach meth- 
od, valid estimates of personality varia- 
bles may be obtained; and (b) to com- 
pare clinical estimates with those based 
upon specific Rorschach scores or pat- 
terns of scores reputed to be descriptive 
of the variable in question. 


total research project. As described in 
the project's first Progress Report and 
Preliminary Fndings (6), the professional 
staff in 1947 consisted of thirty members, 
including two VA psychiatrists and sev- 
eral VA psychologists. Other staff mem- 
bers were psychologists drawn from five 
different universities and several other 
institutions. This resulted in considera- 
ble diversity of background, training, 
and viewpoint. 

The subjects were student VA trainees, 
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already accepted for training by one of 
the cooperating universities. They came 
to Ann Arbor in six successive classes of 
24 each at nine-day intervals, each class 
remaining for seven days. Each class was 
divided into six teams of four students 
each. Each student team was studied in- 
tensively by a staff team of three mem- 
bers. 


B. THE AssEssMENT Project RATING 
SCALES 


A rating scale which would permit of 
clinical judgments on a variety of per- 
sonality attributes was first tried in the 
1946 preliminary program and revised 
for the 1947 program. The ratings on 
this revised rating scale form the basis 
for the assessment research (see Appen- 
dix A). 

This rating scale has three parts: Scale 
A was intended to cover primarily pheno- 
typic variables, descriptive of the subject 
as seen by the staff at the time of assess- 
ment. Including 22 variables, it was con- 
structed with the help of Professor Ray- 
mond Cattell on the basis of his several 
factorial studies of personality ratings 
(2). 

Scale B was intended to provide judg- 
ments of more genotypic variables, i.e., 
evaluative judgments of broader under- 
lying variables, again as seen by the staff 
at the time of assessment. It includes 10 
variables.. 

Scale C includes a list of 11 criterion 
variables on which staff members were 
asked to make predictive judgments rela- 
tive to the future performance of each 
candidate (five years after assessment). 

All ratings were made on an 8-point 
scale, with a theoretical distribution of 
ratings as follows: 


8 Upper 3% of a population of first-year gradu- 
ate students. 
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7 Upper 7% of a population of first-year grad. 
uate students. 

6 Upper 15% of a population of first-year grad. 
uate students. 

5 Upper 25% of a population of first-year grad. 
uate students. 

4 Lower 25%, of a population of first-year grad. 
uate students. 

3 Lower 15% of a population of first-year grad. 
uate students. 

2 Lower 7% of a population of first-year gradu. 
ate students. 

1 Lower 3% of a population of first-year gradu. 
ate students, 


For Scale C, the reference population 
was VA trainees five years after assess. 
ment. Staff members were instructed to 
give anyone who they believed would 
fail in the training program a rating ol 
1, 2, or 3 on Item 42 (“over-all suitability 
for clinical psychology”). 


C. THe CANDIDATE’S SCHEDULE 


Upon arriving in Ann Arbor and reporting to 
the sorority house being used for housing and the 
actual assessment research, the assembled candi- 
dates were given a welcoming speech by the Proj- 
ect Director, who indicated the general nature of 
the program. They were told in a general way 
what was expected of them and assured that 
strict professional confidence would be main- 
tained for all personal material gathered by the 
project. 

That same morning the group was given 3 
battery of objective tests. At lunch, as at all 
meals during the first five days, each team sat 
at a specified table, along with any two staf 
members who would not be evaluating that 
team. In the afternoon of the first day, the clas 
continued taking group tests. Before the end of 
three days, the following tests had been taken: 
the Thematic Apperception Test (10 cards), the 
Project Sentence Completion Test, Miller Analo- 
gies, Thurstone’s Primary Mental Abilities, the 
Allport-Vernon Study of Values, Strong Voca- 
tional Interest Blank, Guilford-Martin Batter) 
(Inventory of Factors STDCR, GAMIN, and 
Personnel Inventory I), and the Minnesota Multi: 
phasic Personality Inventory. During the third 
day or, sometimes, the next day, each subject 
took the Rorschach and the Bender-Gestalt Tesi, 
each at separate sessions. 

The first afternoon, each student had an “In- 
itial Interview” lasting one hour. The inter 
viewer, unknown to the subject, had already 
studied the student's “credentials file,” m& 
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terials he had submitted with his application 
to the university. These included the transcript 
of his college grades, his Civil Service Form 57, 
and letters of recommendation submitted in his 
behalf. This interview was searching but not 
deep lest anxiety be aroused in the subject. 

During the evening of the first day, each 
subject filled out a Biographical Inventory, 
which contained over a hundred multiple-choice 
items about background, personal history, ex- 
periences, and preferences. He also received an 
outline for an autobiography he was to write 
within the next two days. 

On the third day, more objective tests were 
administered including the Kuder Preference 
Record and the ACE Culture Test. The Strong 
was taken twice more: once, he filled it out as 
he thought it would be answered by “women 
in general” and finally as he thought it would 
be answered by “men in general.” These results 
were not made available to the assessment staff. 

Late that afternoon, each subject met his “in- 
tensive interviewer,” the staff member whom he 
would get to know best and the person whose 
job it was to get to know the assigned student 
best. This interview was thorough and probing. 
By the nature of the interviewer's questions, it 
was obvious to each student that the interviewer 
had carefully examined his credentials, auto- 
biography, Biographical Inventory, test scores, 
and the other materials on which the student 
had spent three days. 

The fourth day was free for relaxation and 
sightseeing. The fifth day was spent in a variety 
of situational procedures which were observed 
by staff members. These situations were of three 
kinds: (1) a group, or team situation, in which 
from four to eight students participated jointly, 
(2) dual situations, and (3) individual situations, 
in which each student participated alone, in the 
presence of staff members only. 

On the morning of the last day, the students 
gathered in the group testing room for their 
last written work, which was the completion of 
a sociometric questionnaire concerning his re- 
actions to his classmates, and a rating of himself 
and his three teammates. 

Following this, he had a final individual ap- 
pointment with his interviewer. Usually the stu- 
dent went to this interview expecting to learn 
the staff's opinion of him, but came away with 


relatively little information, such as test scores, 
etc. 


D. THe STAFF SCHEDULE 


_ Much of the staff time allotments can be 
judged from the foregoing description of stu- 
dent activities. Only the more important aspects 
of staff activities not clearly indicated above will 
be discussed in this section. 
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During the earlier portion of the week the 
intensive interviewer prepared for his intensive 
interview with the student assigned to him by 
studying, in order, (1) the credentials file, (2) 
the objective test profiles and answer sheets, (3) 
the autobiography and biographical inventory, 
(4) the projective test protocols and their inter- 
pretations. After each of these four sets of ma- 
terials he made ratings on Scales B and C, Thus, 
each of the four sets of ratings was based on a 
cumulative collection of materials. Each rating 
was filed before making his next rating. 

The projective tests were handled as follows: 
each student took both the Rorschach and the 
Bender-Gestalt Tests individually. Each was in- 
terpreted and summarized by the administrator 
of the test. The Thematic Apperception Test 
and the Sentence Completion Tests were ad- 
ministered as group tests and were analyzed by 
various staff members. No staff member analyzed 
more than one projective test for any one sub- 
ject. In addition to the qualitative interpretation, 
the projective analyst made ratings on the vari- 
ables of Scale B and Scale C on the basis of 
each projective protocol. Thus, four sets of rat- 
ings on Scale B and Scale C each, were based on 
individual projective techniques alone. In addi- 
tion, a “projective integrator” made ratings based 
on all of the projective data collected. 

The interviewer usually spent about four hours 
preparing for the intensive interview. After the 
interview, which lasted about two hours, ratings 
on Scale B and Scale C were made once again. 

At this point, each team of three staff mem- 
bers held a conference to discuss their four stu- 
dents. These preliminary conferences had two 
purposes: the first was to discuss the case and, by 
subjectively pooling the three individual and 
independent ratings on each variable, to arrive 
at a pooled or team rating on each variable; the 
second was to provide for a fuli exchange of 
data and opinion about each case—with both 
the initial and the intensive interviewers re- 
porting on their interviews. 

On the following day, each staff team observed 
its four students in the situational procedures. 
Separate ratings were not made after each situ- 
ation; staff team ratings were made from notes 
the next day on the four students observed. The 
preliminary pooling conference consisted in each 
staff member’s making a set of ratings based on 
his total impression of the student up to and 
including the situation tests. The interviewers 
than held Final Interviews with the students 
whom they had first seen in Intensive Inter- 
views. The substance of this interview was re- 
ported to the other two team members. The 
staff members then examined the tabulations of 
the sociometric questionnaire and studied the 
teammates’ ratings of each other and the stu- 
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dent self-ratings. They also read the character 
sketches to determine how each student appeared 
in the eyes of his teammates and to estimate the 
insight of the author of each sketch. Also, for 
the first time, they were permitted to examine 
all the ratings which had previously been made 
on each subject. Upon the basis of all of these 
data a new rating, the Final Individual Rating 
on each variable, was made. These last were 
taken by each staff team member to the Final 
Team Conference at which the staff team made 
its best judgments about each case in the light 
of the entire week’s assessment. This compre- 
hensive report considered many aspects of the 
person: his emotional life, his intellectual func- 
tioning, his relations with others, his motivation 
for clinical psychology, etc. From a research point 
of view, its most important function was to 
record qualifications about certain ratings: e.g., 
the staff might rate a certain case with the 
understanding that he would soon be undergoing 
psychotherapy. 

The ratings coming out of this conference for 
each student were designated the Final Pooled 
Ratings. Since it is the most comprehensive and 
inclusive set of ratings which was made during 
the program and since it involved the pooled 
judgments of skilled clinicians who had been 
afforded ample opportunity to “get to know each 
subject as God knows him,” it is proposed to use 
those ratings as criterion ratings in the present 
investigation. 


E. Supjects, DATA, AND STATISTICAL 
DESIGN 


In the present study, the subjects con- 
sist of a group of 120 male students as- 
sessed during the 1947 summer program 
in Ann Arbor. All were student VA 
trainees who had already been accepted 
for training by one of the cooperating 
universities. 

The data to be analyzed are drawn 
from three sources: (1) the final pooled 
ratings (criterion ratings) on the ten vari- 
ables comprising Scale B, and on Vari- 
able 42 of Scale C (“over-all suitability for 
clinical psychology’); (2) the ratings on 
the same variables made by the Rorschach 
examiners, based upon information they 
obtained from the Rorschach records; 
and (3) Rorschach scores and patterns 
obtained directly from the Rorschach 
records. 
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Only Scale B variables are being used 
because Rorschach ratings were not al. 
ways made on Scale A during the assess. 
ment program and because it is felt that 
most of the items on Scale C are too spe. 
cifically oriented toward professional 
skills and, hence, remote from the usual 
type of data reportedly gleaned from 
Rorschach results. On the other hand, 
it is rather universally held that the 
Rorschach method taps more covert, 
genotypic personality attributes and this 
is the definition of the Scale B variables. 

Six Rorschach examiners administered 
approximately four records each during 
each week of assessment. Each scored the 
records, following whatever system was 
familiar to him, most using the scoring 
symbols designated by Klopfer (3), al- 
though some used Beck’s categories (1), 
while one used a system combining fea- 
tures of both of these. For the sake of 
uniformity, all scores were studied by 
the writer and converted to those ol 
Klopfer insofar as information available 
in the records permitted. 

It is proposed to correlate the ratings 
made by the Rorschach examiners (here- 
inafter designated “Judges’’) with the 
final pooled ratings (hereinafter desig 
nated “Criterion”) variable-by-variable, 
in order to determine the extent to which 
judgments based on Rorschach informa: 
tion will approximate the criterion judg- 
ments. Since six judges are involved in 
making the Rorschach ratings, the de- 
gree to which the criterion is estimated 
by the individual judges will be studied 
by means of running correlations be 
tween the ratings of each judge and the 
criterion ratings. This will result in an 
array of six correlations for each variable 
and each of such arrays will be tested 
by Rider’s technique (5) for determining 
the homogeneity of a distribution of cor 
relation coefficients. 
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In half of the criterion teams the 
Rorschach examiner was a functioning 
team member. Since this introduces the 
possibility of contamination in the cri- 
terion ratings, the data will be split 
into contaminated and uncontaminated 
groups and separate correlations com- 
puted for each group. Under ordinary 
circumstances of contamination it might 
be expected that the contaminated data 
would show higher correlations than the 
uncontaminated, 

The study of individual Rorschach 
scores and patterns descriptive of the sev- 
eral variables involved will be discussed 
individually. 

Social Adjustment: The scores and 
patterns to be used in predicting this 
variable (Variable 23b) are listed below. 
The rationale behind the selection of 
these particular Rorschach factors lies in 
the interpretative values assigned to 
them in the literature. 


1. Normatively, the number of responses usu- 
ally given to the ten ink-blot designs by healthy, 
normal adults ranges from go to 40. In varying 
degrees of social maladjustment, it has been ob- 
served that the number of responses frequently 
is less than this, often ranging downward from 
25 responses. Miale and Harrower-Erickson (4), 
for example, report a heterogeneous group of 
neurotics as giving an average of only 13 re- 
sponses, only three cases producing more than 25 
responses. Their normal control group averaged 
36 responses by comparison. Decreased produc- 
livity is often interpreted as reflecting one who 
is ill at ease and lacking in drive and spontaneity. 
In the present investigation, therefore, the tend- 
ency to give less than 25 responses will be taken 
as a partial indication of social maladjustment. 

2. Human movement responses are ordinarily 
interpreted as indicating the subject’s degree of 
inner maturity and stability. Normally two or 
more of such responses are expected; less than 
this number suggests a lack of these character- 
istics. Miale’s neurotics, for instance, averaged 
only 1.1 M while the normal controls averaged 
5-5 and all had produced two or more of this 
‘ype of response, Less than two M responses will 
be regarded as a second partial sign of malad- 
Justment. 

3. Animal movement responses, designated as 
FM, are interpreted as more primitive, infantile 
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inner drives in contrast to the more mature hu- 
man movement responses. Normally it is ex- 
pected that in healthy adults, M will exceed 
FM, usually by two to one, In Miale’s neurotic 
group, 67 per cent of the records showed FM to 
be greater than M; the normals produced more 
M than FM in 7o per cent of the records. Indi- 
cations of such an excess of immaturity in the 
inner life in the records of the assessment group 
will be taken, therefore, to be related to social 
maladjustment. 

4. Color responses in general in Rorschach in- 
terpretation are reputedly related to emotional 
responsiveness to affectively toned stimuli from 
the subject’s environment, Form responses are 
interpreted as indicating intellectual control. 
Hence, responses determined by a combination 
of form and color, with form elements pre- 
dominating (FC), are reflective of controlled 
emotionality, hence, emotional adjustment. Two 
or more of such responses are normally ex- 
pected. Neurotics, according to Miale’s study, 
produce an average of only 0.6 FC responses 
while her normal controls averaged 3.1; stated 
differently, 81 per cent of the neurotics gave no 
more than one FC and an equal per cent of 
the normals gave more than one. In fact, 70 
per cent of the neurotics produced no FC re- 
sponses at all. For this reason, less than two 
FC responses will be regarded as indicative of a 
lack of emotional adjustment and more than two 
as suggestive of good emotional adjustment and 
good rapport with others. 

5. Another pattern, related to the previous 
one, but more indicative of emotional imbalance, 
excitability, and egocentricity is given by the 
extent to which responses in which intellectual 
control is subordinated to emotionality (CF) or 
in which intellectual control is completely lack- 
ing (C) exceed the more controlled FC responses. 
Such imbalance and egocentricity is posited as 
indicating social maladjustment. 

6. Normally go to 50 per cent of Rorschach 
responses involve the perception of animal 
figures. A per cent of responses in excess of 50 
per cent is regarded as mental stereotypy. In 58 
per cent of Miale’s neurotic cases animal re- 
sponses constituted over half of the responses; 
only 25 per cent of the normals showed this sign. 
Thus, over 50 per cent animal responses in the 
present records is interpreted as indicating a lack 
of spontaneity associated with social maladjust- 
ment. 

7. Also related to spontaneity in Rorschach in- 
terpretation is the per cent of responses deter- 
mined by form alone. Too high a per cent of 
form responses is interpreted as a restriction of 
spontaneity, with an attempt to substitute intel- 
lectual control for genuine adjustment. Normally, 
go to 50 per cent of the total number of responses 
is expected to be determined by pure form. Fifty- 
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one per cent of Miale’s neurotics showed this 
sign compared with only 20 per cent among the 
normals. Following this, subjects who produced 
over 50 per cent form responses are regarded as 
having one more sign of social maladjustment. 

8. Color shock has long been regarded as in- 
dicative of emotional disturbances and inhibi- 
tions in response to emotionally toned stimuli 
emanating from the environment. Color shock 
was defined in the present study as follows: 

a. A significant delay, over one average devi- 
ation, in time of responding to color cards, es- 
pecially If and VIII. 

b. Exclamations or comments by the subject 
felt to be indicative of disturbance in reaction 
to these cards. 

c. Decline in the quality of responses, de- 
terioration of mode of approach sequence, or 
failure to produce commonly perceived responses 
on the color cards. 

d. Rejection of color cards with no scorable 
response. Using similar criteria for color shock, 
Miale found the reaction to be present in 98 per 
cent of the neurotic records and in only 20 per 
cent of the normal records. 

g. Shading shock, referred to by Klopfer as 
“sexual” shock, are reactions similar to those 
described above to the two highly shaded cards 
(IV and VI) and occasionally to the diffusely 
shaded card VII. Such a reaction is interpreted 
as indicative of considerable anxiety, often re- 
lated to sexual conflicts. This sign was present 
in 81 per cent of Miale’s neurotic records and 
in only 20 per cent of the normal ones. The 
presence of either color shock or shading shock 
is regarded as indicative of poor social adjust- 
ment. 

10. Finally, the complete rejection of any card 
with no scorable response is rarely observed 
among normal, healthy adults. Of Miale’s psycho- 
neurotics, 47 per cent rejected one or more cards, 
the range was from one to seven of the ten 
cards. None of her normal controls showed this 
type of reaction. Rejections in the present study 
will be regarded as another sign of maladjust- 
ment. 


These “signs” of social maladjustment 
will be treated in two different ways. 
The absence of signs will be correlated 
with the criterion ratings to determine 
the extent of concomitant variations be- 
tween them by means of the usual Pear- 
son product-moment method. Second, 
since Miale and Harrower-Erickson (4) 
suggest that the presence of five or more 
signs strongly indicates social maladjust- 


ment, a biserial correlation coefficien 
will be computed, the dichotomous vari. 
ables being designated as records having 
ing less than five signs and records hay. 
ing five or more signs of maladjustment. 

Appropriateness of Emotional Ex. 
pression: This variable is closely related 
to certain of the patterns enumerated 
above, specifically the response categories 
designated FC, CF, and C. The first is 
ordinarily interpreted as indicative of 
appropriate emotional responsiveness 
while the latter two suggest progressively 
more disorganized emotionality. The 
correlations between these individual 
factors will be determined and then con- 
bined to determine the multiple correla. 
tion coefficient obtaining among them 
and the criterion ratings. In addition, 
patterns 4, 5, 7, and 8 noted above under 
Social Adjustment are felt to be related 
to appropriateness of emotional expres 
sion. A biserial coefficient of correlation 
will be run between these patterns and 
the criterion; one dichotomous variable 
will be the presence of less than two ol 
these patterns, the other two or more. 
The former is expected to be related 
to more appropriate emotionality, the 


latter with failures to adapt emotional 


responses to the needs of the situation. 

Characteristic Intensity of Inner Emo- 
tional Tension: Classically, in Rorschach 
interpretation, human movement re 
sponses (M), and more recently animal 
movement responses (FM) and the per 
ception of inanimate objects in motion 
(m) have been regarded as associated 
with the inner life of the individual 
The former is deemed representative 0! 
a more mature, stable stage of develop 
ment in the intrapsychic life while the 
latter two represent progressively a mort 
immature, inhibited, and tension-laden 
state. Hence, these three factors in the 
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records will be correlated individually 
with the criterion ratings on this varia- 
ble (Variable 25) and then combined to 
determine the extent of multiple corre- 
lation among them and the criterion. 
Since a low rating indicates minimum 
tensions and a high rating suggests 
strongly repressed drives resulting in in- 
ner turmoil, a negative correlation co- 
eficient should be expected for the cor- 
relation with M, a positive one with 
m, with FM taking a middle position. 

Another pattern will be studied here 
involving the conception noted above 
that M should exceed FM normally. To 
study this pattern, the writer devised an 
index expression which results in values 
ranging from zero to two as follows: 1— 
(M-FM)/(M+ FM). Scores closer to zero 
would represent the preponderance of 
M in the record while scores approach- 
ing two reflect the excess of FM over M. 
Negative correlation between these val- 
ues and the criterion ratings would be 
expected. 

These same factors will be treated in 
still another way. A biserial correlation 
coeficient will be computed with M = 
FM and M > FM as the dichotomous 
variables. 

Since inner emotional tensions are 
plausibly related to social adjustment 
(Variable 23b), it is proposed that the 
ten patterns used there be employed 
again here. In this instance it would be 
expected that the more signs of malad- 
justment, the higher should be the cri- 
terion rating on inner emotional ten- 
sion. 

Sexual Adjustment: Two Rorschach 
scores are often used to estimate sex- 
ual adjustment developmentally. Animal 
movement responses (FM) are frequently 
interpreted as indicative of more basic, 
instinctual, sexual drives which in adults 
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may be interpreted as a fixation at an 
early level of psychosexual development. 
Another way of gauging this factor is 
in comparison with the more mature in- 
ner drives as reflected by human move- 
ment responses (M). Thus, the num- 
ber of FM responses in the records and 
the above-described index showing the 
relative weights of M or FM will each 
be correlated with the criterion ratings 
on Variable 26. It would be expected 
that each should yield a negative correla- 
tion with sexual adjustment if the in- 
terpretative assumptions indicated above 
are to be retained. 

Sexual shock reactions, defined under 
the section on Social Adjustment, are 
ordinarily interpreted as indicative of 
anxiety over sexual conflicts and, hence, 
of sexual maladjustment. In order to 
assess the value of this pattern a point 
biserial coefficient of correlation will be 
run against the criterion ratings, the di- 
chotomous variables being presence and 
absence, of sexual shock in the records. 

Motivation for Professional Status: 
There are no factors in the Rorschach 
which are presumed to be related to such 
a variable as this one. However, “drive 
for status rewards” is suggestive of Ror- 
schach interpretations involving the 
tendency to over-emphasize responses to 
the whole ink blot (W), frequently re- 
ferred to as “ambition.” Whole responses 
have also been compared with human 
movement responses and norms are 
given in the literature which indicate 
that the healthy balance between these 
two factors is approximately 2:1. When 
W exceeds expectations, “ambition” over 
and above basic creative capacities is in- 
ferred. Some Rorschach examiners also 
feel that this same personality complex 
is indicated by a large number of re- 
sponses to the whole Rorschach experi- 
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ment. Therefore, it is proposed that four 
correlations be run against the criterion 
ratings on Variable 27: (1) number of 
responses utilizing the whole blot (W), 
(2) per cent of responses utilizing the 
whole blot (%W), (3) scores given by the 
formula (W-2M), and (4) total number of 
responses given (R). In each instance 
positive correlation coefficients would be 
expected. 

Motivation for Scientific Understand- 
ing of People: Again, this represents a 
variable which has no clear counterpart 
in Rorschach scores or patterns. It may 
be assumed, however, that scores inter- 
preted as reflecting interest in and em- 
pathy for others, and thinking which is 
in keeping with that of one’s contem- 
poraries might be plausibly related to 
this variable. Content categories involv- 
ing the perception of human beings are 
an example of the former type while 
popular responses, i.e., those commonly 
seen by other persons, are an example of 
the latter. Thus, three correlations will 
be run against the criterion ratings on 
Variable 28: (1) number of responses in 
which the content was human beings 
(H), (2) per cent of responses in which 
the content was human beings (%H), 
and (3) number of popular responses 
(P). In each instance positive correla- 
tions might be expected. 

Insight into Others: There are no par- 
ticular Rorschach factors specifically re- 
lated to this variable insofar as the 
literature is concerned; this is equally 
true of the next variable, Jnsight into 
Himself. However, reference has been 
made from time to time to a more re- 
stricted kind of insightfulness, i.e., in- 
sight into anxieties. The presence of this 
personality characteristic is inferred 
from responses determined in part by 
form in combination with either dif- 


fuse shading and/or surface texture jn 
the ink blots. Diffuse shading (K) is in- 
terpreted as anxiety in the inner life, 
hence, the combination of form and dif. 
fuse shading (FK) represents anxiety 
over which the individual is able to ex. 
ert intellectual control and, hence, anx- 
iety into which he has insight. Surface 
texture, especially in relation to dys. 
phoric concept, is interpreted as anxiety 
in relation to environmental occur. 
rences; the combination of form and tex- 
ture (Fc) suggests situational anxieties 
under intellectual control, or anxieties 
into which the person has _ insight. 
Reasoning similarly, form and color re- 
sponses (FC) may be interpreted as con- 
trolled, or insightful, emotional respon- 
siveness to environmental stimuli. These 
patterns will be used in two ways: (1) 
the sum of responses in which form ele- 
ments were primary and combined with 
textural and diffuse shading deter- 
minants (Fc + FK), and (2) the sum of 
responses in which form and color were 
combined determinants, form predomi- 
nating (FC). These will be employed in 
both Variables 29 and go since they seem 
so closely related by definition and since 
preliminary study reveals the correlation 
between them to be .8o. 

Quality of Intellectual Accomplish- 
ments: Intellectual quality in Rorschach 
work is ordinarily inferred from the 
quality of the subject’s responses. Qual- 
ity is defined as the degree to which a 
given response is apposite to reality and 
is designated as F +. In general, the 
higher the per cent of F + responses, the 
better the quality of the subject’s intel- 
lectual work. In addition to this factor, 
originality of responses of good quality 
(O +) is interpreted as high-level intel- 
lectual ability. Originality is ordinarily 
defined as a response given approxi 
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mately once in a hundred records. A neg- 
atively oriented factor related to this var- 
iable is the percéntage of animal forms 
(% A) seen. This score is interpreted as 
reflecting mental stereotypy when the 
percentage exceeds half of the total 
number of responses given. Thus, four 
correlations will be run against the cri- 
terion ratings on Variable 31: (1) per 
cent of F+ responses, (2) the raw num- 
ber of original responses given (O), (3) 
the number of original responses of good 
quality given (O+), and (4) the per- 
centage of animal responses (%A). In the 
first three instances, positive correlations 
should result; in the latter, the correla- 
tion should be negative. 
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Over-all Suitability for Clinical Psy- 
chology: This variable is obviously 
neither a specific personality characteris- 
tic nor is it one which has ever been 
related to Rorschach results. However, 
one might posit the hypothesis that over- 
all suitability is logically related to over- 
all social adjustment. Working on this 
hypothesis, it is proposed that the ten 
patterns of adjustment used in study- 
ing Variable 23b be used here. It will be 
remembered that the absence of these 
signs was felt to be related to adjust- 
ment, hence, one should expect a posi- 
tive coefficient of correlation in this case 
if the hypothesis is to be retained. 
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CHAPTER III 


RESULTS 


N THIS chapter we compare the cri- 
I terion ratings on each variable with 
(a) the ratings made by Rorschach ex- 
aminers, and (b) the various Rorschach 
scores and patterns. 


A. VARIABLE 23b: SociaAL ADJUSTMENT 


Social adjustment was defined for the 
raters as follows: “How well does he ad- 
just to varied interpersonal situations? 
(Includes sexual adjustment only as it 
affects social adjustment in general.) On 
this variable, the higher the rating, the 
more it indicated a person who “actively 
considers the feelings of others; readily 
gains acceptance in interpersonal rela- 
tionships; maintains a friendly and like- 
able manner.” The lower the rating, the 
more it indicated one who “acts without 


consideration for the feelings of others; 
often rejécted by others; often appears 
aloof, hostile, or irritable.” 

The correlations between the criterion 
and ratings by the Rorschach “judges” 
are shown in the first column of Table 


1. 

On this variable the Rorschach judges 
were able to predict the criterion rat- 
ings to the extent indicated by an r 
of .35. This can be interpreted as indi- 


cating a positive relationship and one 
which is significantly greater than zero 
at better than the 1 per cent level of 
confidence. 

As will be noted in Table 1, there js 
apparently a wide spread of 1's among 
the six judges (from .26 to .63), and one 
is tempted to conclude that Judges A 
and B were able to estimate the criterion 
ratings better than the other four judges. 
However, Rider’s method of testing the 
homogeneity of such a distribution of 
r’s yields a chi-square value of 4.27, 
which with five degrees of freedom gives 
a p of .50, indicating that the array of 
r’s is probably homogeneous. The infer- 
ence of a difference between one judge 
and the others might, therefore, be er- 
roneous. 

In Table 2, while the contaminated r 
appears to be greater, the null hypoth- 
esis that there is probably no difference 
between these correlation coefficients 
may be retained (p = .16). 

The ten Rorschach scores and patterns 
described in the preceding chapter were 
studied in an effort to determine to what 
extent scores alone might be used to pre- 
dict the criterion ratings on “social ad- 
justment.” The absence of these signs 


TABLE 1 
CORRELATIONS BETWEEN RORSCHACH RATINGS AND CRITERION RATINGS 


Obtained correlation coefficients by variables* 


r required at— 


25 


.05 level .or level 


+49 
—.20 


Total .24 


* Correlations which are statistically significant at the .o5 level of confidence or better are printed in italics. P 
** N does not total 120 since one judge rated only three subjects and is excluded from the analysis of individual judges 


but is included in the total correlational analysis. 
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B 16 -63 -37 -60 -47 -09 -67 -50 62 
Cc 19 +32 -56 .26 -10 -27 -65 .46 58 0 
D 21 .28 .28 +23 -38 -38 —.03 .48 -15 -43 
E 19 .26 44 -38 -64 -24 -30 -46 t 
F 19 +27 -I§ —.13 -32 .27 +27 .25 -23 .46 .58 
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TABLE 2 


EFFECT OF CONTAMINATION ON THE CORRELATIONS BETWEEN 


RORSCHACH RATINGS AND CRITERION RATINGS 


Contami- 
nated r 


Uncontami- 
nated r 


Variable 


Critical 
Diff. ratio 


23b -443 .278 


24 500 . 560 
25 348 .163 
26 .118 -403 
27 -469 -193 
28 -541 -497 
29 -174 . 280 
30 -119 . 400 
31 .258 367 
42 . 280 -391 


.165 1.02 -16 
— .060 0.48 
.185 1.06 -I5 
— .285 1.64 .05* 
.276 1.66 +05 
044 +37 
. 106 ©.59 .28 
. 281 1.62 .05* 
—.109 0.62 «87 
0.66 +25 


direction from that hypothesized. 


was correlated with the criterion ratings 
in two different ways; the results are pre- 
sented in Table 3. The Pearson r, com- 
puted from a scattergram, yielded a co- 
efficient of .47; this is somewhat better 
than was obtained for the typical judge’s 
ratings (r = .35). 

The use of these ten factors was stud- 
ied in another way, since in the litera- 
ture it has been suggested that the 
presence of five or more of them is a 
critical number of differentiating ad- 
justment from maladjustment. The com- 
putation of biserial correlation produced 
an = -55, Which is significantly 
greater than zero at better than the 1 
per cent level of confidence. 


B. VARIABLE 24: APPROPRIATENESS OF 
EMOTIONAL EXPRESSION 


This variable is defined as: “How ap- 
propriate are his emotional responses to 
the situation?” The lower the ratings 
the more it indicates an individual who 
“fails to adapt his emotional responses 
to the needs of the situation; shows dis- 
organized or overly constricted emo- 
tional responses.” The higher the rating, 
the more it indicates one who “shows 
emotional responses of a quality and in- 
tensity befitting the situation; reacts 


* While this difference is significant at the .os5 level of confidence, note that it is in the opposite 


spontaneously but appropriately; shows 
well-integrated and flexible patterns of 
emotional behavior.” 

The correlations between the Ror- 
schach judges’ ratings and those of the 
criterion are shown in column two of 
Table 1. The results here are similar to 
those obtained on Variable 23b. The 
r is slightly higher and more in keeping 
with the lower levels of validity obtained 
on many psychological tests; again, the 
total r is positive and significantly 
greater than zero at better than the 1 
per cent level of confidence. 

The chi-square test of homogeneity 
of the distribution of r’s among the six 
judges yielded a chi-square value of 
8.07 and a p of .15. This also may be in- 
terpreted as indicating a homogeneous 
array of r values. 

The investigation with respect to con- 
tamination yielded results which may be 
interpreted as indicating no significant 
contamination effect. 

Various Rorschach scores and _pat- 
terns were studied in relation to this 
variable. First, the three factors FC, CF, 
and C responses were employed. Using 
the following indices, 


1 The criterion ratings to be predicted, 
2 FC responses, 
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3 CF responses, tween ratings by the judges and the cri. 
4 on the following zero order r’s were terion (r = .43), although those involy. 
ing FC and CF are significantly greater 
Ty 01 than zero. The multiple correlation 

Ty .02 ffi n i Vv vi 

O=1.15 6:=2.38 oF = 2.66 0.63 matrix yielded R, 234 = .33. 

Second, the four “signs”: (1) FC te. 
sponses less than two, (2) %F responses 


greater than 50 per cent, (3) presence of 


None of the three zero order r’s ap- 
proaches the size of the correlation be- 


TABLE 3 


COMPARISON OF ESTIMATIONS OF CRITERION RATINGS BY CLINICIANS 
AND FROM RORSCHACH PATTERNS 


Estimate of Cri- Estimate of Criterion 
Variable terion by Judges by Rorschach Indices 
(Table 1) and Patterns 


This 
23b. Social Adjustment 


24. Appropriateness of Emotional Expression wy FC: ar 
CF: .28 484 
25. Intensity of Inner Emotional Tension .24 M: a 


09 


. Sexual Adjustment 


. Motivation for Professional Status 


. Motivation for Scientific Understanding of 
People 


. Insight into Others 


FC: -17 
30. Insight into Himself .27 Fe+FK: 
. Quality of Intellectual Accomplishments 


C 42 Over-all Suitability for Clinical Psychol- 35 52" 
ogy (five years hence) 


* Italicized coefficients are significantly greater than zero at the 5 per cent level or better (ro; =.18, 
=.24). 
Ten patterns of adjustment (see page 5). 

» Five or more of these ten patterns and less than five used to dichotomize the group (page 6). 

® Multiple correlation involving the criterion, FC, CF, and C (page 11). 

4 Four Rorschach patterns of emotionality; group dichotomized on the basis of one or less and two 
or more patterns (page 6). , ce 

* Multiple correlation involving the criterion, M, FM, and m (page 13). 

Index written: 1—(M—FM)/(M+ FM) (page 7). 

* Group dichotomized on the basis of M< FM and M>FM (page 7). 


h Point biserial correlation; group dichotomized on the presence or absence of sexual shock re- 
actions (page 7). 
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color shock reactions, and (4) FC less 
than the sum of CF plus C patterns were 
studied in relation to the criterion rat- 
ings. A biserial r was computed, the 
dichotomous variables being the pres- 
ence of less than two “signs” on the one 
hand and two or more on the other. 
This yielded a biserial r of .48, which 
is significantly greater than zero at bet- 
ter than the 1 per cent level of confi- 
dence. 


C. VARIABLE 25: CHARACTERISTIC INTEN- 
siry OF INNER EMOTIONAL TENSION 


‘This variable is defined as “How in- 
tense is his inner emotional life as in- 
ferred from all available clues?” Here a 
low rating suggests an individual whose 
“inner emotional life is characterized by 
a1 minimum of persistent internal ten- 
sions.” A high rating indicates one who 
“has strongly repressed emotional drives 
resulting in inner turmoil; great inner 
conflict and strong pent-up emotions.” 

Column three of Table 1 presents the 
correlations between ratings of the Ror- 
schach examiners and the criterion rat- 
ings. Here again the total correlation 
is positive and significantly greater than 
zero. The distribution of r’s among the 
six judges is such as to yield a chi square 
ol 6.77, p = .g0 and we may again re- 
gard the distribution as relatively ho- 
mogeneous. 

Reference to Table 2 suggests that 
again suspected contamination appar- 
ently does not affect the results to.a 
significant degree. 

Rorschach factors reputedly related to 
“inner emotional tension” were studied. 
The first of these were the responses in- 
volving (1) human movement percep- 
tions (M), (2) animal movement (FM), 
and (3) perception of inanimate objects 
in motion (m). The following correla- 
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tion matrix was found in this investiga- 
tion: 


1 Criterion ratings 
2 M responses 


3 FM responses 

4 m responses 
—-27 = 49 13, .22 
Ty3 —.04 To, = -31 


| 

M,=5.52 M,=630 M,;= 4.36 M,=—0.85 

1.14 5-11 3-21 1.17 

The correlation observed between the 
criterion ratings and human movement 
responses alone is approximately the 
same as that observed between the Ror- 
schach judges and the criterion (.24). 
Computing the multiple correlation co- 
efficient yields Ry, 234 = .35 Which ap- 
pears to be somewhat better than that 
yielded between judges’ ratings and the 
criterion. 

Other patterns studied included the 
biserial r between patterns of M = FM 
on the one hand and M > FM on the 
other. This yielded practically no cor- 
relation (7, = .0g). These same two 
scores ‘were combined to give an in- 
dex ranging from o to 2 as follows: 
1—(M—FM)/(M+FM). The resulting in- 
dices correlated with the criterion rat- 
ings almost not at all (r = —.06). The 
ten patterns used with Variable 23b were 
used again in this instance, and yielded 
a Pearsonian r of .23; this is about as 
good as the correlation between the rat- 
ings of the judges and the criterion. 


D. VARIABLE 26: SEXUAL ADJUSTMENT 


The general question involved in this 
variable was: “To what degree do his 
sexual needs and activities affect his over- 
all adjustment?” Here, the lower the rat- 
ing, the more it indicated a person whose 
“sexual needs and activities seriously in- 
terfere with his over-all adjustment.” 
The higher the rating, the more likely 
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it indicated an individual whose “sexual 
needs and activities definitely enhance 
his over-all adjustment.” 

Column four of Table 1 presents the 
findings with respect to the ability of 
the Rorschach examiners to predict the 
criterion ratings on sexual adjustment. 

‘The results here are similar to those 
obtained on the previous three variables. 
The total correlation coefficient is posi- 
tive and significantly greater than zero at 
less than the 1 per cent level. The dis- 
tribution of 1’s yields a chi square 
= 1.64, p = .go. In line four of Table 
2 we observe a significant difference be- 
tween the 7’s at the 5 per cent level, but 
not in the expected direction for con- 
tamination to have been the cause, since 
uncontaminated r is significantly 
greater than the contaminated one. 
There is no clear explanation for this 
type of difference in relation to the con- 
tamination hypothesis. 

Rorschach scores selected for study in 
relation to sexual adjustment included 
(1) M responses, (2) FM responses, (3) 
the M: FM ratio and (4) the factor des- 
ignated as “sexual shock.” Briefly, the 
first three scores and patterns yielded es- 
sentially no evidence of correlation be- 
ing of the order of magnitude —.o9, 
—.02, and .o5 respectively. Biserial corre- 
lation, on the other hand, in which the 
dichotomous variables were absence of 
“sexual shock” and presence of “sexual 
shock” produced a point biserial r with 
the criterion ratings of .37 which is sig- 
nificant at less than the 5 per cent level. 
In using this result, one predicts a mean 
criterion rating on sexual adjustment of 
4-33 for the group having little or no 
“sexual shock”; the best prediction for 
the “sexual shock” group is a criterion 
rating of 3.43. These means are signifi- 
cantly different at less than the 1 per 


cent level of confidence. Each of these 
means is also significantly different from 
the mean of the total group (3.82) at 
about the 1 per cent level. 


E. VARIABLE 27: MOTIVATION FOR 
PROFESSIONAL STATUS 


Involved in this variable are ratings 
aimed at stating how strong the subject's 
drive is for the status rewards of a pro- 
fessional career. Low ratings tend to 
indicate low drive, high ratings suggest 
strong drive. 

Column five of Table 1 presents the 
findings indicating the ability of the 
Rorschach judges to estimate the ci- 
terion ratings on this variable. The re- 
sult here, as reflected in the correlation 
coefhcient of .32, indicates positive con- 
comitant variation to a degree signifi- 
cantly greater than zero at better than 
the 1 per cent level of confidence. 

Chi square for the distribution of 
judges’ r’s is 12.36, p < .05. This is the 
only array of r’s which is apparently not 
homogeneous, In this instance the corre- 
lations of Judges B and E are homo- 
geneous and so are those of Judges A, 
C, D, and F. The two groups of homo- 
geneous r’s were then combined by the 
z transformation and produced a dif- 
ference significant at less than the 1 per 
cent level. This suggests the probability 
that Judges B and E were significantly 
better at estimating the criterion ratings 
than were the other four judges. 

Line five of Table 2 gives the only evi- 
dence in favor of the hypothesis that 
there might be significantly greater cor- 
relation between the Rorschach ratings 
and the criterion contaminated by the 
Rorschach examiner’s presence on the 
criterion team than between the Ror 
schach ratings and an uncontaminated 
criterion. 
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The series of Rorschach scores and 
tterns studied relative to this variable 
were: (1) number of responses utilizing 
the whole blot (W), (2) per cent of re- 
sponses using the whole blot (%IW), (3) 
total number of responses given (R), and 
(4) the normative score given by the for- 
mula (W-2M) yields values designed to 
show the relative excess of W over normal 
expectations of M. 

All of these scores yielded negligible 
coefficients. That between W and the 
criterion was —.04; between %W and 
criterion —.07; between R and criterion 
-.12; and (W-2M) correlated with the 
criterion —.17. None of these are posi- 
tive nor are they significantly greater 
than zero at the 5 per cent level of con- 
fidence. 


F. VARIABLE 28: MOTIVATION FOR SCIEN- 
TIFIC UNDERSTANDING OF PEOPLE 


Here the raters are asked to judge 
how strong the subject’s drives are to- 
ward acquiring the facts, theories, and 
skills necessary for the scientific under- 
standing of individual human beings. A 
low rating signifies low drive, a high rat- 
ing a strong one. 

Column six of Table 1 shows the find- 
ings obtained with respect to this vari- 
able. Thé total correlation is .38 and is 
both positive and significantly greater 
than zero at less than the 1 per cent 
level. 


The distribution of r’s among the six 


judges yields a chi square of 0.995, 
p > .g5. This may be interpreted as in- 
dicating a homogeneous distribution of 
1’s. 

The study of the possible effect of a 
contaminated criterion yielded the re- 
sults presented in Table 2. From these 
results it may be concluded that contam- 
ination probably had little or no effect 
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in the direction hypothesized. 

Rorschach factors studied in relation 
to “understanding people” were: (1) 
number of responses in which the con- 
tent was human beings (H), (2) per 
cent of responses involving human be- 
ings (%H), and (3) number of responses 
in which percepts were of the type com- 
monly seen by other persons (P). None 
of these produced correlations signifi- 
cantly greater than zero. 


G. VARIABLE 29: INSIGHT INTO OTHERS 


The task here was to rate the subjects 
on the amount of insight they possess 
into the attitudes, emotions, and motiva- 
tions of others. A low rating suggests an 
individual who “interprets behavior at 
its face value; is insensitive to any but 
gross differences in behavior; does not 
develop any integrated understanding 
of behavior or of people.” A high rating 
indicates the person who “has good 
awareness of underlying dynamics of be- 
havic™;. is sensitive to subtle nuances of 
behavioral responses; is able to develop 
integrated understanding of the be- 
havior of people.” 

The correlational findings relative to 
the ability of the “Rorschachers” to es- 
timate the criterion ratings on Variable 
2g are given in column seven of Table 
1. Here again the total correlation of 
.23 is both positive and significantly 
greater than zero. The distribution of 
r’s among the six judges yields a chi 
square of 1.87 for which p is .go. This 
is interpreted as indicating a homo- 
geneous array of 

The contamination-study results on 
this variable are shown in Table 2. The 
null hypothesis that there is probably 
no difference between these 1’s is re- 
tained, and the idea that contamination 
was a factor operating to increase the 
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degree of correlation is rejected. 
Two Rorschach scores were considered 
sufficiently relevant to this variable to 
warrant study: (1) Sum of responses in 
which form elements were primary and 
combined with textual and diffuse shad- 
ing determinants (Fc + FK) and (2) Sum 
of responses in which form and color 
were combined determinants, form pre- 
dominating (FC). The former values 
when correlated with the criterion rat- 
ings produced a coefficient of .16, the lat- 
ter a coefficient of .17. While both of 
these are positive, they only approach 
the value (.18) which is significantly 
greater than zero at the 5 per cent level. 


H. VARIABLE 30: INSIGHT INTO HIMSELF 


On this variable the judges were asked 
to rate each subject as to “how much 
insight does he have into the underlying 
dynamics of his own attitudes, emotions, 
and motivations?” The higher the rat- 
ing, the more the rater felt the subject 
had insight into himself. 

The correlations obtained in connec- 
tion with this variable are presented in 
column eight, Table 1. The correlation 
coefficient of .27 is positive and signifi- 
cantly greater than zero at less than the 
1 per cent level. The distribution of 
coefficients among the six judges pro- 
duced a chi-square value of 3.903; p 
is .60 and the distribution may be re- 
garded as homogeneous. 

The “contamination” study produced 
a significant difference but, as in the 
case of Variable 26, the direction of the 
difference is the reverse of that expected 
on the basis of contamination effects. 

The same two Rorschach patterns 
which were used in the preceding varia- 
ble were studied in relation to this vari- 
able: (1) (Fe + FK), and (2) FC re- 
sponses. The resulting correlations were 


positive but not significantly greater 
than zero. The obtained r’s were .08 for 
the pattern (Fc + FK) and .17 between 
FC responses and the criterion, the lat- 
ter approaching significance at the 5 
per cent level of confidence. 


I. VARIABLE 31: QUALITY OF INTELLEC- 
TUAL ACCOMPLISHMENTS 


The key question involved in this vari- 
able was: “What is the characteristic 
quality of his intellectual output?” A 
high rating suggests one who “character- 
istically produces intellectual work of 
high quality.” A low rating, one whose 
“intellectual work is characteristically of 
low quality.” 

The correlation observed between the 
judges and the criterion on this varia- 
ble was .33. The breakdown of correla- 
tions by judges is shown in column nine, 
Table 1. The total correlation is posi- 
tive and significantly greater than zero. 
The distribution of r’s among the judges 
produced a chi square of only 3.867 and 
p of .60 which indicates the probability 
that the distribution is homogeneous. 
Again there is no evidence that contam- 
ination had the effect expected. 

The Rorschach scores studied in rela- 
tion to “quality of intellectual accom- 
plishments” included: (1) per cent of 
responses determined by “good’’ form 
(%F+), (2) per cent of responses to 
animal figures (%A), (3) number of orig- 
inal responses (O) and (4) number of 
good quality original responses (O +). In 
all of these, positive correlations are ex- 
pected with the exception of %A which 
would be expected to be negative. The 
correlation coefficients obtained are 
given in Table 3. The correlations in- 
volving %A and O+ approach signifi- 
cance at approximately the 5 per cent 
level of confidence. 
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]. VARIABLE C-42: OvER-ALL SUITABILITY 
ror CLINICAL PsyCHOLOGY 


In the assessment rating scale, Scale 
C is concerned with characteristics more 
directly related to clinical psychology 
per se. It was felt that it would be of in- 
terest to include Item 42 for two reasons: 
(1) because it is the rating relative to over- 
all suitability, and (2) because it was 
used in the Preliminary Report as a cri- 
terion rating. The rating is future- 
oriented in that it required a rating “of 
the subject as he may be five years from 
now.” The reference population is VA 
Clinical Psychologists who have had four 
years of academic and on-the-job train- 
ing (U. S. Civil Service grade-classifica- 
tion of “P-4” or above’). The general 
question was worded thus: “Jn view of 
his assets and liabilities, how well will 
he be able to carry out the several duties, 
diagnosis, therapy and research specified 
for the position of clinical psychologists 
(P-4 and above) in the VA?” The higher 
the rating, the greater his suitability; the 
lower the rating, the less his suitability 
with the added stipulation that a “rat- 
ing of 3 or below means that it is doubt- 
ful if the candidate will ever qualify for 
a P-4 rating.” 

The results of the correlation anal- 
yses for Variable C-42 are presented in 
column ten of Table 1. The total corre- 
lation of .35 is positive and is signifi- 
cantly greater than zero at better than 
the 1 per cent level of confidence. The 
distribution of r’s by judges is homo- 
geneous; chi square being 5.384, 


not appear to be a factor operating to 
increase correlation (Table 2). 
There are, of course, no Rorschach 


"The grade-classification, “P-4,” is now desig- 
nated as “GS-11” in official VA and U. S. Civil 
Service regulations. 
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p = .40. Here again contamination does _ 
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scores or patterns of scores designated as 
related to this variable but since it is 
an over-all rating, plausibly related to 
social adjustment, the ten “signs” em- 
ployed in relation to social adjustment 
on Variable 23b were used again here. 
The obtained correlation coefficient is 
52. 
K. THE JUDGES 

As noted above, there was some ques- 
tion of variability among the judges with 
respect to their ability to estimate the 
criterion ratings. This question was 
answered in part in the studies of the 
distribution of r’s among the six judges 
variable-by-variable. In brief, all of the 
distributions indicated the probability 
of homogeneity with the exception of 
that on Variable 27 where Judges B 
and E appeared to be able to predict to 
a significantly greater degree than the 
other four judges. 

It is the writer’s opinion that signifi- 
cant variability does, in fact, exist among 
the judges even though the above studies 
seem to contraindicate it. Partial justifi- 
cation for such an opinion may be seen 
in the following. The individual judges’ 
predictions may be viewed from another 
standpoint, that of the number of varia- 
bles each was able to predict to an extent 
significantly greater than zero at better 
than the 5 per cent level of confidence 
(reading across Table 1). This analysis 
results in the following information: 

1. Judges A and B were each able to 
predict six of the ten variables at ap- 
proximately the 5 per cent level or less. 

2. Judge E was able to predict five 
variables at approximately this level. 

3. Judge C was able to predict only 
three variables. 

4. Judge D was able to predict only 
one variable. 

5. Judge F was able to predict none. 
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On a priori grounds, there being no 
statistical technique known to the writer 
for analyzing such data, it would appear 
that Judges A, B, and E might be re- 
garded as better judges than C, D, and F, 
who among them made only 4 predic- 
tions at approximately the 5 per cent 
level or better, compared with the 17 
significant predictions made by the 
former group. 

It is obvious that using judges who 
lack the ability to make accurate predic- 
tions would operate as a systematic fac- 
tor in lowering the over-all correlation 
between judges’ ratings and the cri- 
terion. Stated differently, given six 
judges, all of the caliber of Judges A, 
B, and E, we should expect significantly 
higher over-all correlations with the cri- 
teria than those obtained and reported 
above. Other logical arguments in favor 
of the general hypothesis that the 
judges vary greatly in ability are evident 
in even a cursory examination of the ex- 
periential backgrounds of the judges 
themselves. Professional courtesy forbids 
stating here all of these evidences since 
to do so would leave open the possibility 
of identifying the judges and might re- 
flect on them personally. These consid- 
erations suggest an ancillary investiga- 
tion of the importance of the experien- 
tial backgrounds of Rorschach judges. 


L. THE RATING PROCESS 


Certain factors in the rating process it- 
self were studied in an effort to try to 
understand the relative lowness of the 
correlation coefficients between judges’ 
ratings and criterion ratings. 

Among these were included such items 
as systematic errors of measurements (or 
better, errors in rating) with regard to 
the requirement that the raters fit their 
ratings into a predetermined distribu- 


tion (see Appendix A); limited range or 
spread of ratings among the eight pro. 
vided by the scales; and the setting in 
which ratings were made. Some evidence 
was uncovered to support the idea that 
the raters did not produce a “good fit’ 
in distributing their ratings as pre. 
scribed and also that there was a limited 
range of ratings used. While these errors 
may be trivial in their consequences, 
they are reported as sources of error for 
what they are worth. 

The fact that the Rorschach examiners 
found themselves functioning in a situa- 
tion somewhat removed from their usual 
clinical settings, under a certain amount 
of pressure of time, and making unique 
interpretations in the form of ratings on 
a rating scale with which they had had 
limited experience is a factor worthy of 
consideration. At least this suggests that 
the general situation was not an opti- 
mum one for careful, accurate work 
with a complex instrument. 

While nothing can be done about the 
first two factors at the moment, the 
matter of experience with the rating 
scale might suggest the hypothesis that 
with experience in the use of the scale, 
correlation might increase. This possibil- 
ity was examined by dividing the six 
classes in half and comparing the cor- 
relations based on Rorschach estimates 
made during the first three weeks with 
those based on Rorschach estimates 
made during the last three. The results 
of this analysis are presented in Table 
4- Here it will be observed that in gen- 
eral the correlations either remained 
about the same or increased. Thus, in 
Variables 24, 25, 26, and 27 there is no 
marked change; in the remaining vati- 
ables there appears to be a more pro 
nounced change in the direction ex- 
pected. The analyses of the significance 
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TABLE 4 


SIGNIFICANCE OF DIFFERENCE BETWEEN RORSCHACH AND CRITERION CORRELATION COEFFICIENTS 
FOR First THREE WEEKs (I-III) AND Last THREE WEEks (IV—VI) or ASSESSMENT 


Obtained r 
I-III’ IV-VI 


Variable 


Critical 


Diff.. Ratio 


23b 
24 
25 
20 
27 
28 
29 
30 
31 
42 


. 2806 
-0243 
.0320 
.0538 
-0449 
.1627 
.1275 
2985 
. 2168 
.2782 


56* 
-13 
.18 
.30 
-go 
.66 


meme 


* gp, =0.18 based on N (I-II]) =61 and N (IV-VI) =509. 


of the differences between these 1’s re- 
veals that the differences are probably 
“true” ones.in the instances of Variables 
23b, go, and 42 at or about the 5 per 
cent level of confidence. Thus, it would 
appear that there is some reason to be- 
lieve that experience with the rating 


scale led to better predictions. 
Certainly in connection with all these 

considerations, mention must be made 

of the fact that correlations of this type 


are undoubtedly lowered by two other 
important factors: (1) the fact that the 
Rorschach is as yet only a crude in- 
strument and even were it otherwise, one 
would not expect anywhere near a per- 
fect analysis or evaluation through any 
single psychometric technique; and (2) 
the fact that the rater himself, making 
subjective estimates such as he is re- 
quired to do, is no doubt a potent source 
of uncontrolled error of estimate. 
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CHAPTER IV 


SUMMARY, CONCLUSIONS, AND SUGGESTIONS 
FOR FURTHER RESEARCH 


HE problem of this study is the va- 

lidity with which personality attri- 
butes may be estimated from data ob- 
tained by means of the Rorschach meth- 
od of psychodiagnosis. Two aspects of 
the general problem have been consid- 
ered: (1) the validity of estimates made 
by Rorschach examiners based upon 
Rorschach protocols they had obtained; 
and (2) the validity of various Rorschach 
scores and patterns of scores taken di- 
rectly from the Rorschach records and 
regarded as related to the variables in 
question. 

The procedure was to collect criterion 
ratings made by teams of observers who 
had had the opportunity to observe from 
many diverse viewpoints 120 male, VA 
trainee subjects throughout a week of 
intensive assessment activities provided 
in the University of Michigan Assess- 
ment of Clinical Psychologists Project 
during the summer of 1947. The subjects 
were assessed in groups of 24 over a 
period of six weeks. Assessment ratings 
were made on a specially devised rating 
scale at the end of each week of assess- 
ment. In this study only the more covert, 
genotypic variables constituting Scale B 
and the “over-all suitability” rating on 
Scale C were employed. Next, clinical rat- 
ings on the same variables made by 
Rorschach examiners and based on the 
Rorschach protocols and _ psychograms 
were obtained and correlated with the 
criterion ratings. Since the latter ratings 
were made by a total of six Rorschach 
examiners, the correlations between the 
ratings of each individual examiner and 
the criterion were also run in an effort 
to estimate the factors of ability and 
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variability among the examiners. In each 
case, the total. correlation based on all 
120 cases was tested to determine the 
possibility of curvilinearity, with the re. 
sult that all of the correlations appeared 
to be linear; eta never exceeding the 
product moment coefficient at less than 
the 5 per cent level of confidence. 
For each of the 10 variables, Ror- 
schach scores and patterns deemed to be 
of value in estimating the variable in 
question were isolated. These were then 
taken directly from the Rorschach rec- 


ords and correlated with the criterion 


ratings by correlational methods most in 
keeping with the type of data being 
studied; these included product-moment, 
biserial, and multiple-correlation tech- 
niques. 

The results of these investigations in- 
dicated that in all 10 of the variables 
studied, the Rorschach examiners were 
able to estimate the criterion ratings 
positively and to a degree significantly 
greater than zero at approximately the 
1 per cent level of confidence. The range 
of the correlation coefficients was from 
.23 to .43. 

Since Rorschach examiners sat on half 
of the criterion teams, the possibility of 
their having contaminated the criterion 
ratings and thus having increased the 
correlations between their Rorschach 
ratings and the criterion ratings they 
helped to establish, was studied by com- 
paring the “contaminated” correlations 
with the “uncontaminated.” It was con- 
cluded that contamination as defined 
above was not particularly effective in 
spuriously increasing the correlation co- 
efficients. 
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The several Rorschach scores and pat- 
terns studied revealed much variability 
in their capacity to predict the criterion 
ratings, ranging from low negative corre- 
lations to a positive .52. Some of this 
variability is undoubtedly due to the 
fact that in several instances the Ror- 
schach patterns used were only approxi- 
mations of the variables in questicn. 
Thus, Variable 27: Motivation for Pro- 
fessional Status was regarded (in terms 
of the Rorschach) as essentially a ques- 
tion of “drive and ambition”; Variable 
28: Motivation for Scientific Under- 
standing of People was interpreted as 
“interest in and empathy for others’; 
and the two “insight” variables, 29 and 
go, were correlated with Rorschach pat- 
terns suggestive of “insight into anxi- 
eties,” rather than insight into others 
and into himself as defined on the rating 
scale. Correlations significantly greater 
than zero were obtained on the following 
variables: 


1. Variable 2gb: Social Adjustment 


2. Variable 24: Appropriateness of Emotional 
Expression 


3. Variable 25: Characteristic Intensity of In- 
ner Emotional Tension 
4. Variable 26: Sexual Adjustment 


5. Variable 31: Quality of InteHectual Accom- 
plishments 


6. Variable C-42: Over-all Suitability for Clini- 
cal Psychology (defined as “social adjustment” in 
selecting Rorschach patterns). 


In comparing the obtained correla- 
tions in the above instances with the 
correlations between the clinical ratings 
made by the Rorschachers and the cri- 
terion ratings, it was found that in no 
case was there a difference significant at 
the 5 per cent level or less. 

In all the other variables, the clini- 
cally oriented estimates were superior, 
and it is interesting to note that these 
are precisely the variables noted above 
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for which only approximate Rorschach 
patterns could be devised. 

Since it is commonly felt that correla- 
tions should be near .60 in order to 
speak of valid predictions, it would ap- 
pear that the obtained results fall far 
short of indicating validity for either the 
ratings made by the Rorschach examin- 
ers or the estimates based on Rorschach 
scores. On the other hand, when investi- 
gating psychological problems, even very 
small correlations, if statistically signifi- 
cant, are frequently indicative of psy- 
chologically lawful relationships. The 
fact that the correlation coefficients are 
small may only mean that the measure- 
ment situation is contaminated, uncon- 
trolled, not clearly defined, or that meas- 
urement itself is crude owing to the meas- 
uring instrument employed. In the pres- 
ent study, it has been demonstrated 
that many of these latter factors were 
operating to varying degrees. 

While there was in general no statis- 
tically significant degree of variability 
among the predictions of the six Ror- 
schach judges on the several variables, 
evidence in the form of the number of 
variables estimated at less than the 5 
per cent level of confidence indicates 
the probability that three of the Ror- 
schach judges were relatively “good,” 
producing a total of 17 significant pre- 
dictions, in contrast to the remaining 
three who produced a total of only 4 
significant estimates. It is probable that 
the three inferior Rorschach raters 
tended to lower the total correlations. 
Partial evidence would support this split 
of the judges on the basis of their pro- 
fessional and experiential backgrounds. 

In view of the foregoing, and in view 
of the relative inadequacy of any single 
technique such as the Rorschach method 
to supply all the answers, it is not sur- 
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prising that the observed correlations 
were as low as they were, and gratifying 
that they were so consistently positive 
and statistically significant. It is the 
writer’s opinion that had a large part of 
the sources-of-error variance been con- 
trolled, correlations much nearer to a 
level suggestive of valid prediction would 
have resulted. 

On the basis of these results we may 
conclude that: 

1. Clinicians, using Rorschach data, 
are able to make estimates of covert per- 
sonality characteristics with a validity 
coefficient of the order of .35, as repre- 
sented by the present sample of ten at- 
tributes. 

2. Relatively clear personality attri- 
butes and complexes, such as “social ad- 
justment” and “appropriateness of emo- 
tional expression,” can be equally well 
estimated from certain Rorschach scores 
or patterns of scores; those which are 
more specific, such as ‘“‘motivation for 
professional status” or “motivation for 
scientific understanding of people,’’ were 
not as successfully estimated by the score 
approach as they were by the clinical ap- 
proach. 

g. Some Rorschach examiners may be 
adjudged “good” raters while others must 
be regarded as “poor” raters; the differ- 


ence seems to lie in the difference of 
their backgrounds of professional train. 
ing, experience, and interests. 

A number of problems for further in- 
vestigation suggest themselves, a few of 
which may be mentioned here. In the 
first place, a new set of ratings, com- 
pletely “blind,” might be obtained from 
a new panel of Rorschach “experts,” in 
order to evaluate the possible effects of 
personal contact with the subjects. Sec. 
ond, the problem of reliability of Ror- 
schach ratings should be studied by hav. 
ing a number of Rorschach examiners 
rate the same records and by studying 
interjudge agreements. Third, more ex- 
tensive study should be made of the in- 
portance of extent of familiarity with the 
Rorschach method, breadth of general 
clinical experience, working knowledge 
of personality dynamics, and so forth as 
factors correlated with the ability of 
clinicians to estimate personality attri- 
butes from data obtained by the Ror- 
schach and other projective methods. 
Fourth, and related to the preceding sug: 
gestion, work should be done to deter- 
mine the more specific effects of the per- 
sonality of the clinician upon his Ror 
schach-based ratings of personality attri- 
butes. 
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APPENDIX A 


RESEARCH PROJECT ON THE SELECTION OF 
CLINICAL PSYCHOLOGISTS 


REFERENCE POPULATIONS FOR RATINGS 


tion is made that he will have completed the 
training and will have been employed by the 
VA. A rating of 3 or below means that it is 
doubtful that the candidate will ever qualify for 
a P-4 rating with respect to the trait being rated. 
It is possible that some candidates will later 
qualify for a P-4 position even though they rate 
3 or below on a few Criterion Skills. 


For Scales A and B the candidate is rated as 
he is today. The reference group is first-year 
clinical psychology graduate students in universi- 
ties accredited by the APA to offer training in 
clinical psychology. 

For the Criterion Skills (Scale C), the candi- 
date is rated as he will be five years from now. 
The reference population is VA clinical psycholo- RATING SCALE 
gists who have had four years of academic and All ratings will be made in terms of an eight- 


on-the-job training (P-4 and above). In rating point scale illustrated graphically in Figure 1. 
a candidate on the Criterion Skills, the assump- 


Left-Scale A | 3% | 7% | 15%] 254] 254 15%) | Right-Scale A 
Low-Scale - 


Fic. 1. Theoretical distribution of ratings 


RATING SCALE DEFINITIONS 


Scale A (#1-22) 
(For ratings on this scale, 1 = extreme on left side, 8 = extreme on right side.) 


| | | | | J 


l 2 Ko) 4 5 6 7 8 


Note: The attributes in Scale A refer to behavior which can be directly observed on the surface. 
In using this scale, disregard any inferences about underlying dynamics or causes. 


\. Readiness to cooperate —v— Obstructiveness 
Finds ways of cooperating despite difficulties. Frequently is “difficult.” 


2. Consistent —v— Inconsistent 
Behaves in the same general way from day Shows changing and unpredictable moods and 
to day. behavior. 
3. ‘Assertive -v— Submissive 
Attempts to dominate or influence his ac- Tends to let people have their way. 
sociates. 
4. Depressed —v— Cheerful 


Does not smile or laugh easily or frequently. | Generally bubbling over with good cheer. Opti- 
mistic. Enthusiastic. Prone to cheerful, witty re- 
marks. 
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5. 


. Narrow Interests 


10. 


15. 


. Placid 
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Irresponsible Serious 
Not inclined to take responsibilities seri- Accepts appropriate responsibilities toway( 
ously. Thoughtless. Unaware of responsibili- _ others. Shows seriousness of purpose. 


ties of his age. (Do not confuse with #17: 
Conscientious vs. not conscientious.) 


. Gregarious (adient toward people) ~v-Non-gregarious (abient from people) 


Is attracted by and moves toward people in In social groupings, isolates himself. 
social situations. 


. Easily Upset —v— Unshakable 


Easily embarrassed or put off balance. Gets _Self-possessed. Does not lose composure under 
confused in emergency. Blushes, shows ex- emotional provocation. 

citability, becomes incoherent. Momentary 

“nervousness,” not general emotionality. 


Broad Interests 
Uninformed in many areas. Narrow, simple, Talks and acts in an informed way in a wide 
interests. Provincial outlooks. variety of areas. 


. Suspicious Trustful 


Believes rather too quickly that he is being = Accepts statements of others without suspicion 
unfairly treated. Imagines on_ insufficient (not necessarily “gullible.”’) 

grounds that people strongly dislike him. 

Interprets things as having reference to him- 

self when none is intended. Feels persecuted. 


Generous selfish 

Generous and considerate in actions involy- Irritable and resentful or withdrawing from 
ing others at considerable self-sacrifice. situations requiring consideration of others. 
Talks and acts exclusively in terms of own 
interests. 


Silent Talkative 
Says very little. Talks a lot, to everybody. Takes the initiative 
in conversations. When addressed, responds 
quickly. 
Cautious ~v— Adventurous 
Avoids the strange and new. Looks at all Seeks and readily enters into new experiences 
aspects of a situation overcautiously. Does and situations. 
the safe thing. 

. Socially poised and adept -v- Socially clumsy, awkward in social situations 
Polite, poised, and tactful in social situa- | Tactless in social situations. Crude in speech and 
tions. Deals with people gracefully and skill- | manners. Omits proper formalities. Does not meet 
fully. people gracefully. Gauche. 

Note: Applies to relationships with one or 
more people. 

. Rigid -v— Adaptable, flexible 


Sticks to his own ideas and does not adapt Appropriately modifies his behavior to situations. 
to ways of doing things differently from his | Accepts compromises where needed. Is not up: 


own. Does not change and broaden with ex- __ set, surprised, baffled, or irritable if things are 
perience. different from what he expected. 
Dependent —v— Self-sufficient 


Asks for reassurance and support. Attaches Does not lean on others in situations calling for 
himself to individuals and groups instead of _—_ independent action. 
relying on himself. 


-v— Worrying, anxious 
Appears to worry constantly, harried; overtly 
anxious. Shows agitation. 


Outwardly calm and relaxed. 
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Conscientious -v— Not conscientious 


ward Behavior characterized by truthfulness, hon- Not too careful about right and wrong where 
esty, unselfishness. own wishes are concerned, Not particularly just, 
honest, or unselfish. Inclined to somewhat shady 
transactions. 
. Imaginative —v— Unimaginative 
Has a rich and ‘vivid imagination. Thinks Lacks imagination. Approaches problems in a 
of unusual angles and aspects of a ques- literal matter-of-fact fashion. Unresponsive to 
tion. Sensitive to a multitude of emotional the subtleties in a situation. 
and other possibilities not realized by the 
under average person. 
ig. Marked Overt Interest in Opposite Sex -v—Slight Overt Interest in Opposite Sex 
Dates a good deal and/or talks a lot about Talks very littlke about women. Does not use 
opposite sex. Extremely aware of women as _ opportunities for contacts with women. 
women. (Disregard inferred needs or drives.) 
wide 20. Frank ~v— Secretive 
Comes out readily with his real feelings on Keeps his thoughts and feelings to himself. 
various questions. Expresses his feelings, sad 
sicion or gay, easily and constantly. 
21. Dependent Minded -v-Independent Minded 
Intellectually dependent on others. Gener- Thinks things out for himself. Examines every 
ally accepts the opinion of a group or of question persistently and _ individualistically. 
authority without much thought. Makes up his own mind. 
22. Limited overt, emotional expression . -v—Marked overt emotional expression 
from Is apathetic, sluggish, or constricted. Shows hyperkinetic, agitated behavioral responses; 
thers. is overly excitable and over-demonstrative. 
own 
Scale B (# 23-31) 
til (For ratings on this scale, 1 = left side or low, 8 = right side or high.) 
nds 
L 2 3 4 5 6 7 8 
iences Since many of the following attributes (#23-31) are broad factors, it is unlikely that any person will 
fit all of the phrases grouped together at one pole of a given variable. Note also that for some items, 
neither extreme necessarily represents a desirable attribute. 
1S 23a. Ability to develop and maintain warm interpersonal relationships. 
h and 
: hae 23b. Social Adjustment: How well does he adjust to varied interpersonal situations? (includes sexual 
| adjustment only as it affects social adjustment in general.) 
1e oF Acts without consideration for feelings of Actively considers feelings of others; readily 
others; often rejected by others, often ap- gains acceptance in interpersonal relationships; 
pears aloof, hostile, or irritable. maintains a friendly and likeable manner. 
tions. 24. Appropriateness of Emotional Expression: How appropriate are his emotional responses to the 
t situation? 
sare Fails to adapt his emotional responses to the Shows emotional responses of a quality and in- 
needs of the situation; shows disorganized tensity befitting the situation; reacts spontane- 
or overly constricted emotional responses. ously but appropriately; shows well-integrated 
for and flexible patterns of emotional behavior. 
25. Characteristic Intensity of Inner Emotional Tension: How intense is his inner emotional life as 
inferred from all available clues? 3 
Inner emotional life characterized by a Has strongly repressed emotional drives result- 
vertly minimum of persistent internal tensions. ing in inner turmoil; great inner conflict and 


strong pent-up emotions. 
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26. Sexual Adjustment: To what degree do his sexual needs and activities affect his over-all adjust. 


ment? 
His sexual needs and activities seriously in- His sexual needs and activities definitely ep. 
terfere with his over-all adjustment. hance his over-all adjustment. 


27. Motivation for Professional Status: How strong is his drive for the status rewards of a profes. 
sional career? 


28. Motivation for Scientific Understanding of People: How strong are his drives toward acquiring 
the facts, theories, and skills necessary for the scientific understanding of individual human beings? 

29. Insight into Others: How much insight does he have into the attitudes, emotions, and motivations 
of others? 
Interprets behavior at its face value; insensi- Has good awareness of underlying dynamics of 
tive to any but gross differences in behavior; behavior; is sensitive to subtle nuances of be- 
does not develop any integrated understand- _havioral responses; is able to develop integrated 
ing of behavior or of people. understanding of the behavior of people. 


30. Insight into Himself: How much insight does he have into the underlying dynamics of his own 
attitudes, emotions, and motivations? 


31. Quality of Intellectual Accomplishments: What is the semmaneciaics 2 quality of his intellectual 
output? 


Intellectual work is characteristically of low Characteristically produces intellectual work of 
quality. high quality. 


Scale C—Criterion Skills (332-42) 
(For ratings on this scale, 1 = low, 8 = high) 


L | | | 

1 2 ce) 4 5 6 7 8 
Ratings on #33-42 refer to the subject’s performance five years hence. 

What will be his level of competence or skill in the varied aspects of: 

32. Academic Performance: (during next three or four years) How well will he: 


Effectively master course content, successfully complete courses in general psychology, clinical 


psychology, statistics, and related fields; satisfy language requirements for the doctorate; pass 
general examinations. 


33. Clinical Diagnosis: How well will he: 
Recognize dynamics underlying particular responses in both objective and projective tests, 
observe significant interrelationships among responses, relate findings to case history and other 
clinical data. 
Elicit from the patient information required for mental status examinations and case histories; 
ascertain and evaluate attitudes and incidents of psychological significance in the patient. 


Synthesize clinical findings to arrive at an integrated picture of personality development, struc- 
ture, and function. 


34. Individual Psychotherapy: How effectively will he: 
Conduct various types of individual psychotherapy. 


35. Group Psychotherapy: How effectively will he: 
Conduct various types of group psychotherapy. 


36. Research: How well will he: 
Recognize and define important research problems in clinical psychology; critically evaluate and 
apply the research findings of others; think with originality and scientific rigor; employ appto- 
priate experimental design and statistical methods; grasp practical implications of aading. present 
results and conclusions in clear, comprehensive, and well-organized form. 


37. Administration: How well will he: 


Plan and develop psychological programs; make proper administrative decisions; delegate respon- 
sibility appropriately; elicit cooperation from subordinates and superiors; maintain high morale 
among his staff; carry out or direct an appropriate public relations program. 
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_ Supervising Clinical Psychologists: How well will he: 


Carry out the professional supervision of subordinates assigned to him for duty and on-the-job 
instruction; assign their duties; evaluate their performance; instruct them in clinical techniques; 
perform other aspects of in-service training. 


. Teaching Psychology (in a College or University): How well will he: 


Teach college courses in general psychology: motivate students, present concepts and procedures, 
stimulate critical thinking about and integration of course materials, evaluate the products of 
learning. 


. Professional Interpersonal Relations: How well will he: 


Work cooperatively with superiors, subordinates, members of the mental hygiene team, and other 
professional personnel concerned with the patient's welfare; participate in the give and take of 
staff conferences; contribute to group decisions. 


. Integrity of Personal and Professional Behavior: How well will he: 


Recognize and fulfill professional responsibilities; live up to personal commitments; show loyalty 
to professional obligations in the event of outside pressure or promise of personal gain; maintain 
discretion concerning professional matters; appropriately conform with commonly accepted stand- 
ard of moral and social behavior; refrain from coloring facts, evasion, lying, etc. 


. Over-all Suitability for Clinical Psychology: In view of his assets and liabilities, how well will he 


be able to: 


Carry out the several duties, diagnosis, therapy, and research specified for the position of clinical 
psychologist (P-4 and above) in the VA. 
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